The present invention relates to image pickup apparatus in which pixel defects occurring on image are adaptively detected and corrected in accordance with an image scene or an operating state of the apparatus.
The rate of occurrence of defect pixels, i.e., fault pixels is generally higher in such image pickup apparatus as a high-definition electronic camera where a solid-state image pickup device having a large number of pixels is used. Technology for the detection and correction thereof has become indispensable. The yield of solid-state image pickup device becomes higher and the price of the apparatus is greatly reduced by correcting pixels of singularities represented by these fault pixels.
The known technology for electrically correcting such fault pixels includes the following techniques. In particular, Japanese patent laid-open application 55-156482 for example discloses the technique of preparing a memory in which locations of fault pixels occurring peculiarly to each solid-state image pickup device are previously retained at the time of manufacturing the solid-state image pickup device. Such a memory is mounted on the image pickup apparatus such as an image sensor. While continuously surveying output signals from the memory, the fault pixels at the predetermined locations are interpolated for example by means of average values of adjoining pixels.
Further Japanese patent laid-open applications Hei-6-6685 and Hei-9-289614 for example disclose the technique of providing a mode to detect fault pixels under certain image pickup conditions for example at the time of introducing power or at the time of an adjustment involving the operation for shutting out light incident upon the solid-state image pickup device so that, though a memory is provided, information on the detected defects is updated at each occurrence of such certain image pickup conditions.
Further Japanese patent laid-open applications Hei-7-23297 and Hei-9-247548 for example disclose the technique of making correction by determining an arbitrary pixel of an arbitrary image as to its defect based on its correlation with the surrounding pixels during photographing operation of camera.
Furthermore Japanese patent laid-open application 2000-59690 discloses an electronic camera which previously retains location data of fault pixels in a memory and at the same time has a function capable of detecting defects also from an arbitrarily selected taken image in accordance with the exposure time so that it can deal with transient defects occurring at the time of a long exposure.
Now, the one disclosed in the above described Japanese patent laid-open application 55-156482 requires an exclusive memory for each individual sensor, i.e., each image pickup apparatus such as a camera. It cannot be jointly used by separate image pickup apparatus. Further, it cannot deal with the types of defect occurring due to the passage of time after the shipment of the image pickup apparatus. Furthermore, there is also a problem of increase in both price and dissipation power due to the fact that memory size for storing defect locations becomes very large proportionally to the increase in the number of pixels as a result of achieving a high-definition image. Moreover, there is another problem that it cannot deal with the types of defects which occur transiently in a long time exposure or under high temperatures.
The ones disclosed in Japanese patent laid-open applications Hei-6-6685 and Hei-9-289614, on the other hand, are capable of dealing with those defects which occur due to the passage of time. These, however, cannot deal with those transiently occurring defects such as the defects in the case of a temperature hike after continuation of normal image pickup conditions, since the certain image pickup conditions for detecting defects are the conditions outside the normal photographing. Moreover, in the case of detecting defects under certain image pickup conditions, since it is necessary to enter the defect detecting operation mode with limiting to such certain image pickup conditions, there is a problem that images cannot be taken during a long time period of such certain defect detecting operation mode.
Further, the techniques disclosed in Japanese patent laid-open applications Hei-7-23297 and Hei-9-247548 are capable of dealing with changes after the shipment or in temperature, since detection/correction of defects is performed for each image. There are many edge patterns, however, in an arbitrarily selected image and it is difficult to detect defects around edge. An attempt for accurately detecting the defects therefore results in a problem that the size of circuit and program becomes very large. In addition, while a high processing speed is a prerequisite in detecting defects in real time for each taken frame, an exceedingly large circuit and program size slows down the processing speed and makes a real time defect detection and correction thereof impossible.
Furthermore, while the electronic camera disclosed in Japanese patent laid-open application 2000-59690 is adapted to be capable of dealing with transient defects occurring at the time of a long time exposure, it is not uncommon that the occurrence of such transient defects under a long time exposure is concentrated in a small area. In such a case, an accurate detection of the defects is difficult. Also, there is another problem that the technique disclosed therein is incapable of dealing with those defects resulting from changes due to the passage of time after the shipment of the electronic camera.